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Transmission and Receipt of Time-Converging Multi-Media Streams 
[0001] This is a continuation of a patent application entitled "Method and 

Apparatus for Server Broadcast of Time-Converging Multi-Media Streams" having 
Ser. No. 09/320,008 which was filed on May 26, 1999. 
5 Technical Field of the Invention 

[0002] The present invention pertains to the field of playback of streaming 

media (such as audio and audio- visual works) which are retrieved from sources having 
non-deterministic delays such as, for example, servers (such as file servers or 
streaming media servers) broadcasting data via the Internet. In particular, the present 

10 invention pertains to method and apparatus for providing playback of audio or audio- 
visual works received from sources having non-deterministic delays. In further 
particular, the present invention pertains to method and apparatus for providing 
continuous playback of streaming media from sources having non-deterministic delays 
such as, for example, servers (such as file servers or streaming media servers) 

15 broadcasting data via the Internet, an Intranet, or the like. 
Background of the Invention 

[0003] Many digitally encoded audio and audio-visual works are stored as data 

on servers (such as file servers or streaming media servers) that are accessible via the 
Internet for users to download. FIG. 1 shows, in schematic form, how such audio or 

20 audio-visual works are distributed over the Internet. As shown in FIG. 1, media 
broadcast server 2000 accesses data representing the audio or audio- visual work from 
storage medium 2100 and broadcasts the data to multiple recipients 2300i to 2300 n 
across non-deterministic delay network 2200. In this system there are two main 
sources of random delay: (a) delay due to media broadcast server 2000 accessing 

25 storage medium 2100 and (b) delay due to the congestion, interference, and other delay 
mechanisms within network 2200. 

[0004] One well known technique for providing playback of the audio or 

audio-visual work is referred to as batch playback. Batch playback entails 
downloading an entire work and initiating playback after the entire work has been 
30 received. Another well known technique for providing playback of the audio or audio- 
visual work is referred to as "streaming." Streaming entails downloading data which 
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represents the audio or audio-visual work and initiating playback before the entire 
work has been received. 

[0005] There are several disadvantages inherent in both of these techniques. A 

prime disadvantage of batch playback is that the viewer/listener must wait for the entire 
5 work to be downloaded before any portion of the work may be played. This can be 
tedious since the viewer/listener may wait a long time for the transmission to occur, 
only to discover that the work is of little or no interest soon after playback is initiated. 
The streaming technique alleviates this disadvantage of batch playback by initiating 
playback before the entire work has been received. However, a disadvantage of 

10 streaming is that playback is often interrupted when the flow of data is interrupted due 
to network traffic, congestion, transmission errors, and the like. These interruptions 
are tedious and annoying since they occur randomly and have a random duration. In 
addition, intermittent interruptions often cause the context of the playback stream to be 
lost as the viewer/listener waits for playback to be resumed when new data is received. 

15 A further disadvantage of streaming is that a user or client is required to poll for 
additional data according to its rate of use of the data. In this manner, a user or client 
using data at a rapid rate has to make additional requests for data at a higher rate than a 
user or client using the data at a slower rate. 

[0006] A further disadvantage in broadcasting audio or audio-visual works 

20 using prior art methods occurs when clients request data asynchronously from the 
media server. Currently, there are two prior art methods for broadcasting a work to 
multiple clients requesting data at arbitrary times. The first prior art method involves 
re-broadcasting the work at regular intervals. This prior art method is efficient for the 
media server since its storage access patterns and load are basically independent of the 
25 number of clients receiving the audio or audio-visual work. A major problem with this 
prior art method is that clients must join a re-broadcast in the middle of the audio or 
audio-visual work currently being broadcast, or wait for the next re-broadcast to begin 
to view the start of the audio or audio-visual work. 

[0007] The second prior art method initiates a re-broadcast of the audio or 

30 audio-visual work each time a client requests to view the audio or audio-visual work. 
This prior art method has the advantage that client do not have to wait to view the start 
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of work and begin reception immediately. A major problem with this second prior art 
method is that the media server must monitor, track and fulfill the request of each 
client requesting data individually. This causes a dramatic increase in server load 
during heavy use since multiple requests arrive simultaneously, and storage access 
patterns and broadcast load vary widely. As a result, the media server's capacity to 
serve a number of clients in a reasonable time is limited. 

[0008] As one can readily appreciate from the above, a need exists in the art for 

a method and apparatus for providing substantially continuous playback of streaming 
media such as audio and audio-visual works received from sources having non- 
deterministic delays such as a file server broadcasting data via the Internet. In addition, 
a need exists in the art for a method and apparatus for broadcasting streaming media on 
an efficient basis that maximizes broadcast media server capacity. 
Summary of the Invention 

[0009] One or more embodiments of the present invention advantageously 

satisfy one or more of the above-identified needs in the art. In particular, one 
embodiment of the present invention is a method for transmitting a work to a client 
which comprises steps of: (a) saving at least a portion of the work for a first period of 
time; (b) receiving a request for the work from the client; (c) transmitting a time-scale 
modified version of the saved portion of the work for a second period of time, wherein 
the second period of time is substantially equal to a time it takes for the time-scale 
modified version to synchronize with the work, had the work been transmitted from a 
start of the first period; and (d) transmitting the work starting at the synchronized point. 



[00010] FIG. 1 shows, in schematic form, how audio or audio-visual works are 

broadcast from a server (for example, a file server or a streaming media server) to 
recipients over a network such as, for example, the Internet; 

[00011] FIG. 2 shows a block diagram of an embodiment of the present 
invention which provides substantially continuous playback of an audio or audio-visual 
work received from a source having non-deterministic delays such as a server (for 
example, a file server or a streaming media server) broadcasting data via the Internet; 
[00012] FIG. 3 shows, in pictorial form, low and high thresholds used in one 
embodiment of Capture Buffer 400 for the embodiment of the present invention shown 
in FIG. 2; 

[00013] FIG. 4. shows a graph of playback rate versus amount of data in Capture 
Buffer 400 using eqns. (2)-(4) for the embodiment of the present invention shown in 
FIG. 2; 

[00014] FIG. 5. shows, in graphical form, relative amounts of data at an input 
and an output of TSM System 800 in the embodiment of the present invention shown 
in FIG. 2 during time-scale expansion, i.e., slow down of the playback rate of the 
streaming media; 

[00015] FIG. 6. shows, in graphical form, relative amounts of data at an input 
and an output of TSM System 800 compression in the embodiment of the present 
invention shown in FIG. 2 during time-scale compression, i.e., speed up of the 
playback-rate of the streaming media; 

[00016] FIG. 7 shows a block diagram of media server 3000 which re-broadcasts 
an audio or audio-visual work is at regular intervals; 

[00017] FIG. 8 A shows, in graphical form, encoding, transmitting and decoding 

portions of an audio or audio-visual work; 

[00018] FIG. 8B shows, in graphical form, a composition and transmission 
method utilized by Work Stream 3200 to form and transmit a TDM composite signal to 
Multicaster 3300 of embodiment 3000 shown in FIG. 7 of embodiment 3000; 
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[00019] FIG. 9A shows a graph of location of a segment (offset from an origin) 

of the audio or audio-visual work being re-broadcast by embodiment 3000 shown in 
FIG. 7 as a function of time; 

[00020] FIG. 9B shows a graph of work-encoded-data-block number (offset 
from an origin) of the audio or audio-visual work being re-broadcast by embodiment 
4000 shown in FIG. 10 as function of time; 

[00021] FIG. 9C shows a graph of position (offset from an origin) of an audio or 
audio-visual work being received by a media playback device that incorporates storage 
of the audio or audio-visual work, such as, for example a digital VCR, hard-disk based 
VCR, and the like; 

[00022] FIG. 10 shows a block diagram of embodiment 4000 of the present 
invention that transitions asynchronously arriving requests to receive a particular audio 
or audio-visual work to synchronous re-broadcasts of the audio or audio-visual work; 
[00023] FIG. 1 1 A shows a graph of location (offset from an origin) of an audio 
or audio- visual work being re-broadcast by embodiment 5000 shown in FIG. 12 as a 
function of time in accordance with the further aspect of the present invention; 
[00024] FIG. 1 IB shows, in graphical form, encoding portions of a Time-Scale 
Modified audio or audio-visual work to form Time-Scale Modified Leaders; 
[00025] FIG. 12 shows a block diagram of embodiment 5000 of the present 
invention which: (a) transmits Time-Scale Modified Leaders; (b) joins re-broadcast 
offset streams of an audio or audio-visual work; and (c) transmits offset streams of an 
audio or audio-visual work; 

[00026] FIG. 13 shows a graph of location (offset from an origin) in normal and 
Time-Scale Modified versions of offset re-broadcasts of an audio or audio-visual work 
versus time on the horizontal axis; 

[00027] FIG. 14 shows a block diagram of embodiment 21000 of the present 
invention which transmits information relating to the playback speed and/or content of 
media data to clients receiving the media data; and 

[00028] FIG. 15 shows a block diagram of embodiment 24000 of the present 
invention in which information relating to the playback speed and/or content of the 
media data being broadcast to clients is embedded in the media work. 
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Detailed Description 

[00029] FIG. 2 shows a block diagram of embodiment 1000 of the present 
invention which provides substantially continuous playback of an audio or audio-visual 
work received from a source having non-deterministic delays such as a server (for 
example, a file server or a streaming media server) broadcasting via the Internet. As 
shown in FIG.2, streaming data source 100 provides data representing an audio or 
audio-visual work through network 200 to User System 300 (US 300), which data is 
received at a non-deterministic rate by US 300. Capture Buffer 400 in US 300 receives 
the data as input. In a preferred embodiment of the present invention, Capture Buffer 
400 is a FIFO (First In First Out) buffer existing, for example, in a general purpose 
memory store of US 300. 

[00030] hi the absence of delays in arrival of data at US 300 from network 200, 
the amount of data in Capture Buffer 400 ought to remain substantially constant as a 
data transfer rate is typically chosen to be substantially equal to a playback rate. 
However, as is well known to those of ordinary skill in the art, pauses and delays in 
transmission of the data through network 200 to Capture Buffer 400 cause data 
depletion therein. Data depletion in Capture Buffer 400 occurs because, 
simultaneously, data is input thereto from network 200 while data is output (for 
example, at a constant rate) therefrom to satisfy data use requirements of Playback 
System 500. As should be clear to those of ordinary skill in the art, if data transmitted 
to US 300 is delayed long enough, data in Capture Buffer 400 will be consumed, and 
Playback System 500 must pause until a sufficient amount of data has arrived to enable 
resumption of playback. Thus, a typical playback system must constantly check for 
arrival of new data while the playback system is paused, and it must initiate playback 
once a sufficient amount of new data is received. 

[00031] hi accordance with the present invention, data input to Capture Buffer 
400 of US 300 is buffered for a predetermined amount of time, which predetermined 
amount of time typically varies, for example, from one (1) second to several seconds. 
Then, Time-Scale Modification (TSM) methods are used to slow the playback rate of 
the audio or audio-visual work to substantially match a data drain rate required by 
Playback System 500 with a streaming data rate of the arriving data representing the 
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audio or audio-visual work. As is well known to those of ordinary skill in the art, 
presently known methods for Time-Scale Modification ("TSM") enable digitally 
recorded audio to be modified so that a perceived articulation rate of spoken passages, 
i.e., a speaking rate, can be modified dynamically during playback. During Time-Scale 
expansion, TSM System 800 requires less input data to generate a fixed interval of 
output data. Thus, in accordance with the present invention, if a delay occurs during 
transmission of the audio or audio-visual work from network 200 to US 300 (of course, 
it should be clear that such delays may result from any number of causes such as delays 
in accessing data from a storage device, delays in transmission of the data from a 
media server, delays in transmission through network 200, and so forth), the playback 
rate is automatically slowed to reduce the amount of data drained from Capture Buffer 
400 per unit time. As a result, and in accordance with the present invention, more time 
is provided for data to arrive at US 300 before the data in Capture Buffer 400 is 
exhausted. Advantageously, this delays the onset of data depletion in Capture Buffer 
400 which would cause Playback System 500 to pause. 

[00032] As shown in FIG. 2, Capture Buffer 400 receives the following as input: 
(a) media data input from network 200; (b) requests for information about the amount 
of data stored therein from Capture Buffer Monitor 600; and (c) media stream data 
requests from TSM System 800. hi response, Capture Buffer 400 produces the 
following as output: (a) a stream of data representing portions of an audio or audio- 
visual work (output to TSM System 800); (b) a stream of location information used to 
identify the position in the stream of data (output to TSM System 800); and (c) the 
amount of data stored therein (output to Capture Buffer Monitor 600). It should be 
well known to those of ordinary skill in the art that Capture Buffer 400 may include a 
digital storage device. There are many methods well known to those of ordinary skill 
in the art for utilizing digital storage devices, for example a "hard disk drive," to store 
and retrieve general purpose data. There exist many commercially available apparatus 
which are well known to those of ordinary skill in the art for use as a digital storage 
device such as, for example, a CD-ROM, a digital tape, a magnetic disc. 
[00033] As further shown in FIG. 2, and in accordance with the present 
invention, TSM Rate Determiner 700 receives the following as input: (a) a signal (from 



Capture Buffer Monitor 600) that represents the amount of data present in Capture 
Buffer 400; (b) a signal (output, for example, from Playback System 500 or from 
another module of US 300) that represents a current data consumption rate of Playback 
System 500; (c) a low threshold value parameter (TL which is described in detail 
below) for the amount of data in Capture Buffer 400; (d) a high threshold value 
parameter (TH which is described in detail below) for the amount of data in Capture 
Buffer 400; (e) a parameter designated Interval_Size; and (f) a parameter designated 
Speed_Change_Resolution. In response, TSM Rate Determiner 700 produces as 
output a rate signal representing a TSM rate, or playback rate, which can help better 
balance the data consumption rate of Playback System 500 with an arrival rate of data 
at Capture Buffer 400. 

[00034] hi a preferred embodiment of the present invention, TSM Rate 
Determiner 700 uses the parameter Interval Size to segment the input digital data 
stream in Capture Buffer 400 and to determine a single TSM rate for each segment of 
the input digital stream. Note, the length of each segment is given by the value of the 
Interval_Size parameter. 

[00035] TSM Rate Determiner 700 uses the parameter 
Speed_Change_Resolution to determine appropriate TSM rates to pass to TSM System 
800. A desired TSM rate is converted to one of the quantized levels in a manner which 
is well known to those of ordinary skill in the art. This means that the TSM rate, or 
playback rate, can change only if the desired TSM rate changes by an amount that 
exceeds the difference between quantized levels, i.e., Speed_Change_Resolution. As a 
practical matter then, parameter Speed_Change_Resolution filters small changes in 
TSM rate, or playback rate. The parameters Interval_Size and 
Speed_Change_Resolution can be set as predetermined parameters for embodiment 
1000 in accordance with methods which are well known to those of ordinary skill in 
the art or they can be entered and/or varied by receiving user input through a user 
interface in accordance with methods which are well known to those of ordinary skill 
in the art. However, the manner in which these parameters are set and/or varied are not 
shown for ease of understanding the present invention. 



[00036] As still further shown in FIG. 2, TSM System 800 receives as input: (a) 
a stream of data representing portions of the audio or audio- visual work (output from 
Capture Buffer 400); (b) a stream of location information (output from Capture Buffer 
400) used to identify the position in the stream of data being sent, for example, a 
5 sample count or time value; and (c) the rate signal specifying the desired TSM rate, or 
playback rate (output from TSM Rate Determiner 700). 

[00037] In accordance with the present invention, TSM System 800 modifies the 
input stream of data in accordance with well known TSM methods to produce, as 
output, a stream of samples that represents a Time-Scale Modified signal. The Time- 

10 Scale modified output signal contains fewer samples per block of input data if Time- 
Scale Compression is applied, as shown in FIG. 6. Similarly, if Time-Scale Expansion 
is applied, the output from TSM System 800 contains more samples per block of input 
data, as shown in FIG. 5. Thus, TSM System 800 can create more samples than it is 
given by creating an output stream with a slower playback rate (Time-Scale 

15 Expanded). Similarly, TSM System 800 can create fewer samples than it is given by 
creating an output stream with a faster playback rate (Time-Scale Compressed), hi a 
preferred embodiment of the present invention, the TSM method used is a method 
disclosed in U.S. Patent No. 5,175,769 ( the '769 patent), which '769 patent is 
incorporated by reference herein, the inventor of the present invention also being a 

20 joint inventor of the '769 patent. Thus, the output from TSM System 800 is a stream 
of samples representing portions of the audio or audio-visual work, which output is 
applied as input to Playback System 500. Playback System 500 plays back the data 
output from TSM System 800. There are many well known methods of implementing 
Playback System 500 that are well known to those of ordinary skill in the art. For 

25 example, many methods are known to those of ordinary skill in the art for 
implementing Playback system 500, for example, as a playback engine. 
[00038] In accordance with the present invention, the stream of digital samples 

output from TSM System 800 has a playback rate, supplied from TSM Rate 
Determiner 700, that provides a balance of the data consumption rate of TSM System 

30 800 with the arrival rate of data input to US 300. Note that, in accordance with this 
embodiment of the present invention, the data consumption rate of Playback System 
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500 is fixed to be identical to the data output rate of TSM System 800. Thus, when a 
playback rate representing Time-Scale Expansion is output from TSM Rate Determiner 
700 and applied as input to TSM System 800, the number of data samples required per 
unit time by TSM System 800 is reduced in proportion to the amount of Time-Scale 
Expansion. A reduction in the number of data signals sent to TSM System 800 slows 
the data drain-rate from Capture Buffer 400 and, as a result, less data from Capture 
Buffer 400 is consumed per unit time. This, in turn, increases the amount of playback 
time before a pause is required due to emptying of Capture Buffer 400. 
[00039] As one of ordinary skill in the art should readily appreciate, although the 
present invention has been described in terms of slowing down playback, the present 
invention is not thusly limited and includes embodiments where the playback rate is 
increased in situations where data arrives in Capture Buffer 400 at a rate which is faster 
than the rate at which it would be consumed during playback at a normal rate. In this 
situation, the playback rate is increased and the data is consumed by TSM System 800 
at a faster rate to avoid having Capture Buffer 400 overflow. 

[00040] As one of ordinary skill in the art can readily appreciate, whenever 
embodiment 1000 provides playback rate adjustments for an audio-visual work, TSM 
System 800 speeds up or slows down visual information to match the audio in the 
audio-visual work. To do this in a preferred embodiment, the video signal is "Frame- 
subsampled" or "Frame-replicated" in accordance with any one of the many methods 
known to those of ordinary skill in the prior art to maintain synchronism between the 
audio and visual portions of the audio-visual work. Thus, if one speeds up the audio 
and samples are requested at a faster rate, the frame stream is subsampled, i.e. frames 
are skipped. 

[00041] Although FIG. 2 shows embodiment 1000 to be comprised of separate 

modules, in a preferred embodiment, Playback System 500, Capture Buffer Monitor 
600, TSM Rate Determiner 700, and TSM System 800 are embodied as software 
programs or modules which run on a general purpose computer such as, for example, a 
personal computer. It should be well known to one of ordinary skill in the art, in light 
of the detailed description above, how to implement these programs or modules in 
software. 



[00042] As should be clear to those of ordinary skill in the art, embodiments of 
the present invention include the use of any one of a number of algorithms for 
determining the playback rate to help balance the rate of data consumption for playing 
back the audio or audio-visual works with the rate of data input from network 200 

5 having non-deterministic delays. In one embodiment of the present invention, the 
playback rate is determined to vary with the fraction of Capture Buffer 400 that is filled 
with data. For example, for each 10% decrement of data depletion, the playback rate is 
reduced by 10%, except when the input data contains an "end" signal. It should be 
clear to those of ordinary skill in the art how to modify this algorithm to achieve any of 

10 a number of desired balance conditions. For example, in situations where a delay 
duration can vary drastically, a non-linear relationship may be used to determine the 
playback rate. One non-linear function that may be used is the inverse tangent 
function, hi this case, 

Playback Rate = tanh" 1 ((2 * #samples_in_buffer/elements_in_buffer)) - 1) (1) 

15 [00043] where #samples_in_buffer is the number of samples of data in Capture 
Buffer 400 and elements_in_buffer is the total number of samples of data that can be 
stored in Capture Buffer 400. 

[00044] In a preferred embodiment of the present invention, a low threshold (T L ) 

value and a high threshold (T H ) value are be used to construct a piece-wise graph of 
20 playback rate versus amount of data in Capture Buffer 400. FIG. 3 shows, in pictorial 
form, how T L and T H relate to the amount of data in Capture Buffer 400. These 
thresholds are used in accordance with the following set of equations: 

For 0 <= X <= T L Playback Rate = Scale * tanh" 1 ((X - T L )/ T L ) (2) 
For T L < X < T H Playback Rate = 1 .0 (the default playback rate) (3) 
25 For T H <= X <= Max Playback Rate = Scale * tanh' 1 ((X - T H )/(Max - T H )) (4) 

where Scale is arbitrary scale factor. 

[00045] FIG 4. shows a graph of playback rate versus amount of data in Capture 
Buffer 400 using eqns. (2)-(4). From FIG. 4, one can readily appreciate that for small 
deviations from an ideal amount of data in Capture Buffer 400 (origin 0 in FIG. 4), 
30 changes in the playback rate are linear; however, larger deviations generate a more 
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pronounced non-linear response. Further, changes in the amount of data in Capture 
Buffer 400 which remain between low threshold level T L and high threshold level T H 
do not cause any change in playback rate. The parameters T L and T H can be set as 
predetermined parameters for embodiment 1000 in accordance with methods which are 
5 well known to those of ordinary skill in the art or they can be entered and/or varied by 
receiving user input through a user interface in accordance with methods which are 
well known to those of ordinary skill in the art. However, the manner in which these 
parameters are set and/or varied are not shown for ease of understanding the present 
invention. 

10 [00046] As should be clear to those of ordinary skill in the art, the inventive 
technique for providing substantially continuous playback may be combined with any 
number of apparatus which provide time-scale modification and maybe combined with 
or share components with such systems. 

[00047] It should be clear to those of ordinary skill in the art, in light of the 
15 detailed description set forth above, that in essence, embodiments of the present 
invention (a) determine a measure of a mismatch between a data arrival rate and a data 
consumption rate and (b) utilize time-scale modification to adjust these rates. Various 
embodiments of the invention utilize various methods (a) for determining information 
which indicates the measure of the mismatch and (b) for determining a playback rate 
20 which enables time-scale modification to adjust for the mismatch in a predetermined 
amount. 

[00048] In light of this, in another embodiment of the present invention, the 

playback system determines that there is a data mismatch because it determines a 
diminution in the arrival of data for playback or subsequent distribution. In response, 

25 the playback system sends this information to the TSM Rate Determiner to develop an 
acceptable playback rate. For example, the playback rate may be reduced by a 
predetermined amount based on an input parameter or in accordance with any one of a 
number of algorithms that may be developed by those of ordinary skill in the art. 
[00049] Embodiments of the present invention are advantageous in enabling a 

30 single-broadcast system utilizing a broadcast server to provide a single broadcast 
across one or more non-deterministic delay networks to multiple recipients, for 



example across the Internet and/or other networks such as Local Area Networks 
(LANs) and Wide Area Networks (WANs). In such a single-broadcast system, the 
path to each recipient varies. In fact, the path to each recipient may dynamically 
change based on loading, congestion and other factors. Therefore, the amount of delay 

5 associated with the transmission of each data packet that has been sent by the broadcast 
server varies. In prior art client-server schemes, each recipient has to notify the 
broadcast server of its readiness to receive more data, thereby forcing the broadcast 
server to serve multiple requests to provide a steady stream of data at the recipients' 
data ports. Advantageously, embodiments of the present invention enable the 

10 broadcast server to send out a steady stream of information, and the recipients of the 
intermittently arriving data to adjust the playback rate of the data to accommodate the 
non-uniform arrival rates, hi addition, in accordance with the present invention, each 
of the recipients can accommodate the arrival rates independently. 
[00050] Another aspect of the present invention advantageously involves 

15 simplification of a transmission protocol used for communication of streaming media 
between a client and, for example, a server such as a media or broadcast server, hi 
accordance with this additional aspect of the present invention, an inventive 
transmission protocol comprises the client's sending a data transmission rate to the 
media or broadcast server. In response, the server transmits data to the server 

20 substantially at that rate, hi one embodiment of the present invention, the data 
transmission rate is in the form of a playback rate for a work. In this case, the server 
adjusts its data transmission rate in a manner which is well known to those of ordinary 
skill in the art so that the amount of data received by the client substantially matches 
the client's playback rate for the work. Thus, for embodiments of this aspect of the 

25 present invention, the broadcast server need not change its distribution rate unless and 
until a new request is received from the client. As one can readily appreciate, the 
inventive transmission protocol is advantageous because its use reduces: (a) a network 
protocol bandwidth required for streaming by substantially reducing repeated requests 
for data from the client and (b) the number of messages the server must process. 

30 [00051] Another aspect of the present invention pertains to media broadcasting 
wherein media or broadcast servers begin broadcasts of a particular work (for example, 
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the day's news) at regular time intervals, for example, every 5 minutes. In accordance 
with an embodiment of the present invention, a client that sends a request to view or 
listen to the particular work is" sent a stream of the particular work (substantially 
immediately) which is closest (in the temporal sense) to the beginning of a broadcast of 
the particular work (in a manner that will be described in detail below) rather than 
waiting for a re-broadcast to begin or joining an in-progress broadcast. After a 
transition period (to be described in detail below), the client joins one of the regular 
broadcasts and receives data therefrom. Advantageously, in accordance with the 
present invention, there is a reduction in client wait time and a reduction in client load 
for the media or broadcast servers. Although embodiments of the present invention are 
described below in the context of broadcasting data for ease of understanding the 
invention, it should be understood that the present invention is not thereby limited. In 
fact, among other things, embodiments of the present invention can also be applied to 
accessing data as well. 

[00052] FIG. 7 shows a block diagram of embodiment 3000 of a media server 
which re-broadcasts an audio or audio-visual work at regular intervals. As shown in 
FIG. 7, Storage Device 3100 is a storage device of a type which is well known to those 
of ordinary skill in the art. Storage Device 3100 stores a representation (preferably a 
digital representation, or any representation that can be converted to a digital 
representation in accordance with methods which are well known to those of ordinary 
skill in the art) of an audio or audio-visual work or data of interest to a user (such as a 
stream of stock quotes, market data, advertisements, and so forth). Storage Device 
3100 receives, as input, data requests from Work Streamer 3200, and provides, as 
output, the data requested. 

[00053] As further shown in FIG. 7, Work Streamer 3200 receives as input: (a) a 
code (for example, a number) which represents a desired Re-broadcast Interval ("RBI") 
(from Re-broadcast Interval Determiner 3700); (b) a code (for example, a number) 
which represents a desired number of Time-Division Multiplexed (TDM) channels 
(from Re-broadcast Interval Determiner 3700); (c) a code (for example, a number) 
which represents the duration of the audio or audio-visual work being re-broadcast 
(from Re-broadcast Interval Determiner 3700); and (d) data from Storage Device 3100. 
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Work Streamer 3200 produces, as output, a Time-Division Multiplexed composite 
signal (described in detail below), which output is applied as input to Multicaster 3300. 
As will be described in detail below, in accordance with the preferred embodiment of 
the present invention, Work Streamer 3200 creates numerous re-broadcasts of the 

5 audio or audio-visual work by sending appropriately interleaved segments of the work 
in the form of the Time-Division Multiplexed signal to Multicaster 3300. 
Advantageously, in accordance with the preferred embodiment of the present 
invention, data accesses to Storage Device 3100 are organized by Work Streamer 3200 
to reduce seek time, decrease latency, and increase throughput by interleaving and 

10 caching accesses to data representing the audio or audio- visual work. It should be clear 
to those of ordinary skill in the art that embodiments of this aspect of the present 
invention are not limited to generating the composite Time-Division Multiplexed 
signal and include embodiments where the data for the various re-broadcasts are each 
generated from a separate signal. 

15 [00054] As still further shown in FIG. 7, Re-broadcast Interval Determiner 
("RBID") 3700 receives, as input, (a) a code (for example, a number) which represents 
the duration of an audio or audio/visual work and (b) a code (for example, a number) 
that represents the number of re-broadcast offset channels (to be described in detail 
below). RBID 3700 produces, as output: (a) a code (for example, a number) which 

20 represents a desired Re-broadcast Interval ("RBI") (sent to Work Streamer 3200); (b) a 
code (for example, a number) which represents a desired number of Time-Division 
Multiplexed (TDM) channels (sent to Work Streamer 3200); (c) a code (for example, a 
number) which represents the duration of the audio or audio-visual work being re- 
broadcast (sent to Work Streamer 3200). In accordance with the present invention, 

25 RBID 3700 computes the RBI by applying one of a number of formulas, fn the 
preferred embodiment of the present invention, the following formula is used: 
RBI = Duration of audio or audio-visual work / No. of re-broadcast offset channels (5) 
[00055] As yet still further shown in FIG. 7, Multicaster 3300 receives, as input: 

(a) a data stream (the Time-Division Multiplexed composite signal) from Work 

30 Streamer 3200 and (b) client information (for example, control and destination) from 
Request Processor 3500. Multicaster 3300 produces, as output, data (for example, 
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message packets) directed toward particular clients for re-broadcast on a network such 
as the Internet, WAN, LAN, etc. In accordance with the present invention, Multicaster 
3300 manages a list of all clients that should receive data from particular portions of 
the TDM composite signal in accordance with any one of a number of methods which 
are well known to those of ordinary skill in the art. Then, whenever the particular 
portion of data in the TDM composite signal is received from Work Streamer 3200, 
Multicaster 3300 sends the particular portion of data to all clients (recipients) in the list 
who are to receive the particular portion of data (also known as multicasting). Many 
methods for broadcasting a portion of data from a data stream (for example, a Time- 
Division Multiplexed composite signal) to multiple recipients are well known to those 
of ordinary skill in the art. Control information from Request Processor 4500 is used 
in accordance with methods that are well known to those of ordinary skill in the art to 
modify the list of recipients, for example, to add a recipient, or to remove a recipient 
from the list of destinations when the recipient no longer desires to receive data from 
the server. 

[00056] As yet again still further shown in FIG. 7, Request Processor 3500 
receives, as input, requests for data from clients connected via Network 3990 (for 
example, an Internet, WAN, LAN, or the like). In response, Request Processor 3500 
produces, as output, information identifying the client and appropriate re-broadcast 
control information such as, for example, "request data that is identified by an 
appropriate data identifier," "disconnect," and other messages that are used to obtain 
data from embodiment 3000. It should be clear to those of ordinary skill in the art, that 
such information identifying the client and appropriate re-broadcast control 
information may be obtained: (a) by dialogs between the client and Request Processor 
3500 in accordance any one of many methods that are well known to those of ordinary 
skill in the art including, without limitation, by use of forms that are contained on web 
pages that are transmitted to the client over Network 3990 in accordance any one of the 
many methods that are well known to those of ordinary skill in the art. 
[00057] Although FIG. 7 shows embodiment 3000 to be comprised of separate 
modules, in a preferred embodiment, the modules are embodied as software programs 
or modules which run on a general purpose computer such as, for example, a personal 
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computer. It should be well known to one of ordinary skill in the art, in light of the 
detailed description above, how to implement these programs or modules in software. 
[00058] Conversely, components of embodiment 4000 may exist in separate 
locations connected to one another by a network or any other communication means 
(where the use of the term means is used in the broadest sense possible). 
[00059] hi accordance with this embodiment of the present invention, an audio 

or audio-visual work is encoded into data that is later decoded to recreate the original 
audio or audio-visual work. Those of ordinary skill in the art should readily appreciate 
that the amount of data that represents a particular portion of the audio or audio-visual 
work can be transmitted, re-broadcast, and/or accessed from Storage Device 3100 in a 
time interval that is significantly less than the playback time interval of the particular 
portion. FIG. 8A shows the playback time of segments (SegO, Segl, SegN) of an 
original audio or audio-visual work, plotted along time axis 3001. The segments are 
encoded as data in accordance with methods that are well known to those of ordinary 
skill in the art, and the transmission times for the encoded data blocks (DO, Dl, 
DN) which correspond to the segments (SegO, Segl, SegN) are shown along time- 
axis 3002. The time of receipt of the transmitted encoded data blocks (DO, Dl, 
DN) are plotted along time axis 3003. As should be clear to those of ordinary skill in 
the art, the encoded data blocks (DO, Dl, DN) are received after an arbitrary 
transmission delay through Network 3990. Lastly, after decoding, the segments (SegO, 
Segl, SegN) of the reconstructed audio or audio-visual work are plotted along 
time-axis 3004. Many methods are well known to those of ordinary skill in the art for 
encoding and decoding audio or audio-visual works. 

[00060] Since, as discussed above, the transmission time of data that represents 

a particular portion of an audio or audio-visual work is generally smaller than the 
playback time interval of the particular portion, two or more audio or audio-visual 
works can be transmitted across a network by interleaving or Time-Division 
Multiplexing (TDM) the data representing the two audio or audio-visual works. Many 
methods are well known to those of ordinary skill in the art for interleaving and Time- 
Division Multiplexing data representing audio or audio-visual works during 
transmission across a network, hi the preferred embodiment of the present invention, 
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Time-Division Multiplexing is used to transmit data to Multicaster 3300. In particular, 
Work Streamer 3200 accesses data from Storage Device 3100, and outputs a TDM 
stream of data to Multicaster 3300. 

[00061] In accordance with the preferred embodiment of the present invention, 
the audio or audio-visual work is divided into segments that are encoded as data for 
efficient storage and transmission. The encoded data representing an interval of the 
media work will be referred to as a "work-encoded-data-block." FIG. 8B shows, in 
graphical form, a composition and transmission method utilized by Work Stream 3200 
to form and transmit a TDM composite signal to Multicaster 3300 of embodiment 
3000 shown in FIG. 7 of embodiment 3000. As shown in FIG. 8B, time axis 3010 
shows the playback time of segments (SegO, Segl, Seg5) of the audio or audio- 
visual work being re-broadcast. SegO corresponds to the first segment of the audio or 
audio-visual work. Thus, time axis 3010 corresponds to a re-broadcast of the work that 
starts at the origin of the time axis (hence its designation as offset 0). Below time axis 
3010 is shown the TDM transmit interval of a work-encoded-data-block that 
corresponds to the segment below which it appears (i.e., the time it takes to transmit 
the data). As discussed above, the time to transmit the corresponding work-encoded- 
data-block is less than the playback time of the segment. As further shown in FIG. 8B, 
time axis 3020 shows the playback time of segments (Seg5, SegO, Seg4) of the 
audio or audio-visual work being re-broadcast. SegO corresponds to the first segment 
of the audio or audio-visual work. Thus, time axis 3020 corresponds to a re-broadcast 
of the work that starts offset from the origin of the time axis by one Re-broadcast 
Interval ("RBI") (hence its designation as offset 1). Below time axis 3020 is shown the 
TDM transmit interval of a work-encoded-data-block which corresponds to the 
segment below which it appears (i.e., the time it takes to transmit the data). However, 
it is offset in time by an amount equal to the time it takes to transmit the work- 
encoded-data-block corresponding to segment SEGO from the offset 0 data stream. 
[00062] Thus, in accordance with the present invention, the re-broadcasts of the 

audio or audio-visual work are labeled: offset 0 stream, offset 1 stream, offset 2 stream, 
and so forth (along time axes 3010 - 3060) and the various offset streams represent re- 
broadcasts of the audio or audio-visual work at regular time intervals, which are 
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referred to as Re-broadcast Intervals ("RBI"). That is, the starting times for the 
particular audio or audio-visual work being re-broadcast are offset at regular intervals, 
RBI, with the start of the re-broadcast of the audio or audio-visual work being denoted 
by SegO in each of the offset data streams shown in FIG. 8B). 
5 [00063] In accordance with the preferred embodiment of the present invention, 

Work Streamer 3200 transmits composite signal 3065 (as Shown in FIG. 8B) to 
Multicaster 3500. As shown in FIG. 8B, composite signal 3065 is a TDM signal that is 
made up of TDM frames (this type of Time-Division Multiplexing is well known to 
those of ordinary skill in the art and many methods are well known to those of ordinary 

10 skill in the art for forming such a signal). As further shown in FIG. 8B, each TDM 
frame of composite signal 3065 comprises a work-encoded-data-block from each of the 
offset streams 0-5, wherein each of the work-encoded-data-blocks is offset in time for 
the time it takes to transmit a work-encoded-data-block. As one can readily appreciate, 
each TDM frame thereby comprises a work-encoded-data-block from each of the re- 

15 broadcasts in the appropriate time slot within the TDM frame. As should be well 
understood by those of ordinary skill in the art, the work-encoded-data-blocks are 
created by Work Streamer 3200 at the appropriate TDM transmit interval (as indicated 
on FIG. 8B) by sending appropriate signals to Storage Device 3100 at regular intervals. 
Note that the TDM transmission interval for each of re-broadcast offset streams 0-5 

20 occurs at a unique time offset from the start of the TDM Frame. Advantageously, this 
enables multiple re-broadcasts of the audio or audio-visual work to be sent Multicaster 
3300 in a TDM format. Further in accordance with the preferred embodiment of the 
present invention, the use of a TDM composite signal enables interleaved data access 
to Storage Device 3100 to provide greater performance in many storage devices of the 

25 type that are well known to those of ordinary skill in the art. It should also be noted 
that, even though it has been depicted in this manner for sake of ease of understanding 
the present invention in FIG. 8B, the transmission time required to send a work- 
encoded-data-block during a particular time slot may not consume the entire time slot 
interval. 

30 [00064] It should be understood that although the preferred embodiment of the 
present invention utilizes a TDM composite signal, the invention is not thereby 
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restricted and includes embodiments wherein other methods utilizing multiple streams 
and/or multiple storage devices, for example, one stream and perhaps one storage 
device for each re-broadcast, can be employed to send data from Work Streamer 3200 
to Multicaster 3300. It should be clear that the TDM composite signal can have a 
number of channels that is bounded by the ability of the system to broadcast to clients 
without the clients noticing a lapse in transmission (of course this cannot account for 
nondeterministic delays in the network). If a larger number of channels is needed to 
handle the predetermined broadcasts than can be handled by the system without lapses, 
one could, for example, create multiple TDM composite signals to handle the extra 
load. 

[00065] FIG. 9A shows a graph of location of a segment (offset from an origin) 
of the audio or audio-visual work being re-broadcast by embodiment 3000 shown in 
FIG. 7 as a function of time. As shown in FIG. 9A, during transmission of data at 
normal playback rates, the locations of segments of the audio or audio-visual work 
being broadcast as a function of time form a line having slope which represents the 
playback rate (a "normal" playback rate corresponding to a slope of 1) and an intercept 
on the time axis at the re-broadcast start time of the particular re-broadcast of the audio 
or audio- visual work. Data streams 3110 - 3140 shown in FIG. 9 A have the same 
playback rates (and therefore the same slope), but are offset from one another since 
they have different re-broadcast start times. Note that at any particular time along the 
horizontal time axis multiple segments from different portions of the audio or audio- 
visual work are being re-broadcast simultaneously. This is seen by drawing a vertical 
line that intersects the horizontal time axis at a particular time. 

[00066] As further shown in FIG. 9A, at 2 time units from the start of the re- 

broadcast of data stream 3110 (offset 0 data stream), client AO sends a request to 
embodiment 3000 to begin viewing the particular audio or audio-visual work being re- 
broadcast. Client AO must wait for the next re-broadcast to begin before receiving data 
(as shown in FIG. 9A, the next re-broadcast starts at the temporal location denoted by 
RBI). Thus, client AO must wait 8 time units before receiving media or other data 
requested from data stream 3120 (offset 1 data stream). Similarly, if client Al sends a 
request to begin viewing the particular audio or audio-visual work being re-broadcast 7 
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time units after the start of the re-broadcast of data stream 3110 (offset 0 data stream), 
client Al has to wait 3 time units before receiving media or other data requested from 
data stream 3120 (offset 1 data stream). 

[00067] FIG. 10 shows a block diagram of embodiment 4000 of the present 
invention that transitions asynchronously arriving requests to receive a particular audio 
or audio-visual work to synchronous re-broadcasts of the audio or audio-visual work. 
First, for ease of understanding the present invention, a general description of how 
embodiment 4000 operates is given with reference to FIG. 9A. In accordance with the 
present invention, embodiment 4000 causes the client to be joined to a data stream 
whose re-broadcast start time is closest (temporally) to the arrival of the client's 
request. Thus, in accordance with the present invention, for client AO, embodiment 
4000 determines that the arrival time of the request to begin viewing is closest to the 
re-broadcast start time of data stream 3110 (note that the re-broadcast time of data 
stream 3110 has already occurred). Embodiment 4000 then begins re-broadcasting 
data to client AO at 5/3 the normal rate, i.e., at an accelerated rate. In accordance with 
the present invention, transmission at the accelerated rate enables client AO to "catch 
up" to the normal re-broadcast location in the audio or audio-visual work 5 time units 
after the re-broadcast start time of data stream 3110. In response to receiving data at 
the accelerated rate, the client (or the client's server) automatically initiates playback at 
an appropriate rate to keep its arrival or capture buffer from overflowing, for example, 
in accordance with aspects of the present invention that have been described in detail 
above. In an alternative embodiment of the present invention, instead of having the 
client determine the accelerated rate required by its playback system to avoid an 
overflow of the data received, the accelerated rate is transmitted to the client by 
embodiment 4000 when transmission is starts. 

[00068] Next, in accordance with the present invention, whenever embodiment 
4000 determines that the stream of data being re-broadcast to client AO relates to the 
same playback position as data stream 3110, embodiment 4000 sends client AO data at 
the normal rate from data stream 3110. Advantageously, in accordance with the 
present invention, overhead on embodiment 3000, and the corresponding components 
of embodiment 4000, is reduced since, as will be explained in detail below, the client 
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now receives data from Multicaster 4300 and no additional accesses to Storage Device 
4100 or Work Streamer 4200 are required to provide data to the client. Additionally, 
as will be described in detail below, once the transition to the offset stream has 
occurred, the client no longer consumes resources Variable Rate Broadcaster 4600. 

5 [00069] In a similar manner to that described above, in accordance with the 

present invention, for client Al, embodiment 4000 determines that the arrival time of 
the request to begin viewing is closest to the re-broadcast start of data stream 3120 
(note that the re-broadcast time of data stream 3120 has yet to occur). Embodiment 
4000 then begins re-broadcasting data to client Al at 5/8 the normal rate, i.e., at a 

10 reduced rate. In accordance with the present invention, transmission at the reduced rate 
enables client Al to reach the normal re-broadcast location in the audio or audio-visual 
work 5 time units after the re-broadcast start time of data stream 3120. In response to 
receiving data at the reduced rate, the client (or the client's server) automatically 
initiates playback at an appropriate rate to keep its arrival or capture buffer from 

15 emptying, for example, in accordance with aspects of the present invention that have 
been described in detail above. In an alternative embodiment of the present invention, 
instead of having the client determine the reduced rate required by its playback system 
to avoid emptying, the reduced rate is transmitted to the client by embodiment 4000 
when transmission is started. 

20 [00070] Next, in accordance with the present invention, whenever system 4000 

determines that the stream of data being re-broadcast to client Al relates to the same 
playback position as data stream 3120, embodiment 4000 sends client Al data at the 
normal rate from stream 3120. Advantageously, in accordance with the present 
invention, overhead on embodiment 3000, and the corresponding components of 

25 embodiment 4000, is reduced since, as will be explained in detail below, the client now 
receives data from Multicaster 4300 and no additional accesses to Storage Device 4100 
or Work Streamer 4200 are required to provide data to the client. Additionally, as will 
be described in detail below, once the transition to the offset stream has occurred, the 
client no longer consumes resources Variable Rate Broadcaster 4600. 

30 [00071] We now return to a detailed description of embodiment 4000. As 

shown in FIG. 10, Storage Device 4100 is a storage device of a type which is well 
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known to those of ordinary skill in the art. Storage Device 4100 stores a representation 
(preferably a digital representation or any representation that can be converted to a 
digital representation in accordance with methods which are well known to those of 
ordinary skill in the art) of an audio or audio-visual work or data of interest to a user 
(such as a stream of stock quotes, market data, advertisements, and so forth). Storage 
Device 4100 receives, as input: (a) data requests from Work Streamer 4200; data 
requests from Variable Rate Broadcaster 4600. Storage Device 4100 provides, as 
output, the data requested. 

[00072] As further shown in FIG. 10, Work Streamer 4200 receives as input: (a) 

a code (for example, a number) which represents a desired Re-broadcast Interval 
("RBI") from Re-broadcast Interval Determiner 4700; (b) a code (for example, a 
number) which represents a desired number of Time-Division Multiplexed (TDM) 
channels; (c) a code (for example, a number) which represents the duration of the audio 
or audio visual work being re-broadcast; and (d) data transmitted from Storage Device 
4100. Work Streamer 4200 produces, as output: (a) a Time-Division Multiplexed 
composite signal as described above (this is applied as input to Multicaster 4300) and 
(b) a stream of information that provides the playback position and time offset of each 
time-offset re-broadcast stream of the work (this is applied as input to Slope/Rate 
Determiner 4400). 

[00073] As still further shown in FIG. 10, Re-broadcast Interval Determiner 
("RBID") 4700 receives, as input, (a) a code (for example, a number) which represents 
the duration of an audio or audio/visual work being re-broadcast and (b) a code (for 
example, a number) that represents the number of re-broadcast offset channels, and 
produces, as output, data representing the duration of the re-broadcast interval ("RBI"). 
In accordance with the present invention, Re-broadcast Interval Determiner 4700 
computes the RBI by applying one of a number of formulas. In the preferred 
embodiment of the present invention, the following formula is used: 
RBI = Duration of audio or audio-visual work / No. of re-broadcast offset channels (6) 
[00074] As yet still further shown in FIG. 10, Slope Rate Determiner ("SRD") 
4400 receives as input: (a) data representing start times for each offset data stream of 
the audio or audio visual work being re-broadcast from Work Streamer 4200; (b) 



-24- 

arrival times and client identification information from Request Processor 4500; and 
(c) a set of parameters representing maximum allowable Time-Scale Modification rates 
(or slopes). SRD 4400 produces as output: (a) client identification information for the 
client requesting the data (applied as input to Variable Rate Broadcaster 4600 "VRB 
4600" and Multicaster 4300); (b) an identification of the re-broadcast offset data 
stream the client will be synchronized or merged with (applied as input to VRB 4600); 
(c) an indication of the time it will take before the synchronization or merge takes place 
("duration to intercept") (applied as input to VRB 4600 and Multicaster 4300); and (d) 
a slope, which represents the playback rate (this slope or playback rate is applied as 
input to VRB 4600). In accordance with the present invention, SRD 4400 determines 
the re-broadcast offset data stream whose start time is temporally closest to the arrival 
time of the client's request by computing the distance forward and backward in time to 
the previous offset stream start time and the next offset stream start time (with respect 
to the arrival time), and choosing the smaller of the forward and backward times. 
Next, SRD 4400 computes a client playback slope that is greater than 1.0 if "catching- 
up" to a future playback intercept position in an offset stream already started, or a 
client playback slope that is less than 1.0 to "slow-down" to a future playback intercept 
position in an offset stream which will start in the future. The actual slope can be 
determined by a number of factors such as the utilization of VRB 4600 (if a higher 
slope is used, there is a greater load since data must be sent faster), and the maximum 
allowable Time-Scale Modification rate. In most cases slope values will be between 
1/2 and 2.0, and can be calculated: (a) by computing a desired playback position 
change per unit time; (b) by accessing a pre-computed look-up table; or (c) any other 
method of choosing a reasonable slope, such as, by client input relating to the speed of 
the "catch-up." The "duration to intercept" is calculated by subtracting the time that 
data transmission to the client is initiated from the time interval at which the playback 
positions of VRB 4600 and the target offset stream are identical. The "duration to 
intercept" information is used to signal VRB 4600 when it must end transmission to 
the client and to signal Multicaster 4300 when it must initiate transmission of the 
appropriate offset stream. 
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[00075] As yet again still further shown in FIG. 10, VRB 4600 receives as input: 

(a) data from Storage Device 4100; (b) a slope or playback rate from SRD 4400; (c) 
client identification information for the client requesting the data from SRD 4400; (d) 
an identification of the re-broadcast offset data stream the client will be synchronized 
or merged with; and (e) "duration to intercept" from SRD 4400. VRB 4600 obtains the 
specified data received from Storage Device 4100 and broadcasts it to the identified 
client at the specified rate for an amount of time equal to the "duration to intercept" and 
then stops sending data for that client. VRB 4600 produces as output a stream of data 
sent to the identified client via a network such as the Internet or Intranet and so forth. 
[00076] As yet still further shown in FIG. 10, Multicaster 4300 receives as input: 

(a) a data stream (the Time-Division Multiplexed composite signal) from Work 
Streamer 4200; (b) client information (control and destination) from Request processor 
4500; (c) client identification information for the client requesting the data from SRD 
4400; and (d) "duration to intercept" information from SRD 4400. Multicaster 4300 
produces, as output, data (for example, message packets) directed toward particular 
clients for clients for re-broadcast on a network such as the Internet, WAN, LAN, etc. 
In accordance with the present invention, Multicaster 4300 manages a list of all clients 
that should receive data from particular portions of the TDM composite signal in 
accordance with any one of a number of methods which are well known to those of 
ordinary skill in the art. Then, whenever the particular portion of data in the TDM 
composite signal is received from Work Streamer 4200, Multicaster 4300 sends the 
particular portion of data to all clients (recipients) in the list who are to receive the 
particular portion of data (also known as multicasting). Many methods for 
broadcasting a portion of data from a data stream (for example, a Time-Division 
Multiplexed composite signal) to multiple recipients are well known to those of 
ordinary skill in the art. Control information from Request Processor 4500 are used in 
accordance with methods that are well known to those of ordinary skill in the art to 
modify the list of recipients, for example, to add a recipient, or to remove a recipient 
from the list of destinations when the recipient no longer desires to receive data from 
the server. The "duration to intercept" and client information from SRD 4400 is used 
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to notify Multicaster 4300 when clients previously receiving data from VRB 4600 
should begin receiving data from one of the offset streams. 

[00077] Lastly, Request Processor 4500 receives, as input, requests for data from 
clients connected via a network (for example, an Internet, WAN, LAN, or the like). In 
response, Request Processor 4500 produces, as output, information identifying the 
client and the appropriate re-broadcast control information such as, for example, 
"request data that is identified by an appropriate data identifier," "disconnect," and 
other messages that are used to obtain data from embodiment 4000. It should be clear 
to those of ordinary skill in the art, that such information identifying the client and 
appropriate re-broadcast control information may be obtained: (a) by dialogs between 
the client and Request Processor 4500 in accordance any one of many methods that are 
well known to those of ordinary skill in the art including, without limitation, by use of 
forms that are contained on web pages that are transmitted to the client over a network 
in accordance any one of the many methods that are well known to those of ordinary 
skill in the art. 

[00078] Although FIG. 10 shows embodiment 4000 to be comprised of separate 
modules, in a preferred embodiment, the modules are embodied as software programs 
or modules which run on a general purpose computer such as, for example, a personal 
computer. It should be well known to one of ordinary skill in the art, in light of the 
detailed description above, how to implement these programs or modules in software. 
[00079] Conversely, components of embodiment 4000 may exist in separate 
locations connected to one another by a network or any other communication means 
(where the use of the term means is used in the broadest sense possible). 
[00080] To better understand the operation of embodiment 4000 described 

above, FIG. 9B shows a graph of work-encoded-data-block number (offset from an 
origin) of the audio or audio-visual work being re-broadcast by embodiment 4000 as 
function of time. As shown in FIG. 9B, during transmission of data at normal playback 
rates, the work-encoded-data-blocks of the audio or audio-visual work being re- 
broadcast as a function of time, form a line having slope 1 and an intercept on the time 
axis at the re-broadcast start time of the particular broadcast of the work. As further 
shown in FIG. 9B, data streams 3110 and 3120 are offset from one another, i.e., they 
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are data streams having different "re-broadcast start-times." However, data streams 
3110 and 3120 have the same playback rate, i.e., the same slope in FIG. 9B, but 
different intercepts on the time axis corresponding to their different start times. As 
discussed above, streams transmitted at a normal rate have slope 1. 
[00081] As further shown in FIG. 9B, the work-encoded-data-blocks sent by 
VRB 4600 of embodiment 4000 are identical to the work-encoded-data-blocks sent by 
Multicaster 4300 of embodiment 4000, but are simply broadcast with a different time 
interval between the work-encoded-data-blocks. This means that VRB 4600 sends the 
same work-encoded-data-blocks that Multicaster 4300 sends, but adjusts the time 
interval between transmissions of the work-encoded-data-blocks in order to "catch-up 
to" or "wait-for" the stream of data blocks sent by Multicaster 4300. If the client 
request is received between (n)RBI and (n+l/2)RBI (less than the half-way point, in 
time, between the nth and the (n+l)st re-broadcast start time), the inter-transmission 
interval is shortened to catch-up to an intersection point in the previous offset stream 
(note the data transmitted for path 3111 has the same work-encoded-data-blocks as 
path 3110, but spaced closer together in time). If the client request is received between 
(n+l/2)RBI and (n+l)RBI (more than the half-way point, in time, between the nth and 
the (n+l)st re-broadcast start time), the inter-transmission interval is lengthened to wait 
for an intersection point with the next offset stream (note the data transmitted for path 
3121 has the same work-encoded-data-blocks as path 3120, but spaced further apart in 
time). Note that, in accordance with the present invention, only the initial portion of 
the audio or audio-visual work is broadcast by VRB 4600, and thus VRB 4600 can 
cache the work-encoded-data-blocks from the initial portion of the work to further 
reduce the number of accesses to Storage Device 4100. There are many methods and 
apparatus that are well known to those of ordinary skill in the art for caching data, such 
as, for example, SRAM, DRAM, or small capacity hard disks. 

[00082] Then, in accordance with the present invention, once embodiment 4000 

(or the transmitting server) determines that the stream of data being broadcast to client 
Al is accessing the same playback position during the same time interval as stream 
3120, the server sends the client data at the normal rate from stream 3120 and the 
overhead on the media server is reduced. In the preferred embodiment the duration to 
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intercept is used to identify the time at which the stream from VRB 4600 and 
Multicaster 4300 will intercept and when the responsibility of transmitting data to the 
client should transition from VRB 4600 to Multicaster 4300. 

[00083] Although aspects of the present invention have been described in the 
context of aligning or synchronizing to one of several, offset, re-broadcast data 
streams, it should be clear that the present invention is not limited to time alignment of 
re-broadcasted works. In fact, embodiments of the present invention may also be used 
to align or synchronize with (catch up), for example, live broadcasts by storing or time 
shifting an audio or audio-visual work that is broadcast only once. For example, in 
digital VCRs or televisions that contain digital storage for spooling live broadcasts, a 
user may watch a live broadcast and invoke a "Pause" function (by, for example, 
pressing a "pause and record" button) to stop playback and initiate recording of the 
one-time broadcast audio or audio-visual work. In response, the one-time broadcast 
audio or audio-visual work is recorded from the point where the Pause function was 
invoked to the end of the audio or audio-visual work. As should be clear, this enables 
a user to turn to other tasks. After the user is ready to return to the work, playback is 
resumed at the location where the "Pause" was initiated by playing the recorded copy 
of the one-time broadcast. This playback is said to be "time-shifted" since the 
playback time differs from the one-time broadcast time. In prior art devices, there is no 
mechanism to catch up to the one-time broadcast without deleting or skipping some 
portion of the time-shifted copy of the audio and/or audio-visual work in the broadcast. 
However, embodiments of the present invention, can be used to catch-up to the live 
one-time broadcast by computing the playback rate required to catch-up in a 
predetermined interval, such as a commercial break, program boundary, or the like. In 
this manner users watching a time-shifted version can "catch-up" to a live broadcast 
after they have paused their viewing devices. 

[00084] FIG. 9C shows a graph of position (offset from an origin) of an audio or 
audio-visual work being received by a media playback device that incorporates storage 
of the audio or audio-visual work, such as, for example a digital VCR, hard-disk based 
VCR, and the like. As shown in FIG. 9C, a one-time live broadcast (4001) is being 
viewed as it is broadcast. At time Tp, the user invokes the Pause function. As 
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previously described, the one-time live broadcast is recorded after invoking the Pause 
function. At time Tr, the user resumes viewing at the same location in the work where 
the Pause function was invoked. However, the user is behind the live broadcast. 
Using the inventive method and apparatus previously described with respect to 
embodiment 4000, the user specifies the amount of time desired to transition back to 
the one-time live broadcast (received by Request Processor 4500), and apparatus, 
similar to Slope/Rate Determiner 4400, computes a Time-Scale Modification Rate, or 
Playback Rate, that will transition the user to the one-time live broadcast during the 
specified interval. The intercept interval computed by embodiment 4000 determines 
the time at which the media playback device may discontinue recording of the one-time 
live broadcast, since after the intercept interval, the user will be viewing the one-time 
live broadcast as it is received. As shown in FIG. 9C, time Tv marks the time at which 
the user will again be viewing the live broadcast. The time interval from Tr to Tv 
defines the transition period during which the user will be watching a Time-Scale 
compressed (or speeded-up) version of the recorded material. The time interval from 
Tp to Tv defines the interval of the one-time live broadcast which must be recorded in 
order to provide a seamless transition to the live-broadcast from a time-shifted viewing 
reference. 

[00085] Note that upon intersecting with the one-time live broadcast, there is no 
longer a need to continue recording the work, and this process is stopped. The ability 
of embodiment 4000 to merge with or "catch-up" to a one-time live broadcast of a 
work from a time-shifted copy of the work that has been recorded significantly reduces 
the amount of the work which must be stored or recorded. This reduction further 
reduces the required storage resources required. 

[00086] In accordance with a further aspect of the present invention, system 

overhead for serving requests from clients that arrive during intervals between the start 
of re-broadcasts of a particular audio or audio-visual work is further reduced. In 
accordance with this further aspect of the present invention, portions of the audio or 
audio-visual work being broadcast are Time-Scale Modified at two rates: one rate is 
faster than normal speed and one rate is slower than normal speed. These slow-rate 
and fast-rate broadcast portions of the audio or audio-visual work are re-broadcast 
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during an interval from the start of re-broadcast of the work to a point X during the 
broadcasting of the work, which point X is a function of the re-broadcast interval and 
the amount of Time-Scale Modification performed. These portions will be referred to 
as Time-Scaled Leaders. FIG. 1 1 A shows a graph of location (offset from an origin) of 
an audio or audio-visual work being re-broadcast by embodiment 5000 shown in FIG. 
12 as a function of time in accordance with the further aspect of the present invention. 
As shown in FIG. 11 A, embodiment 5000 (TSM System 5400 of embodiment 5000 
shown in FIG. 12) time compresses the first 10 time units of the audio or audio-visual 
work and creates a new data stream 3210 which has duration 5 units and a playback 
rate of 2. Note that only 10 time units of the beginning of the original audio or audio- 
visual work are time-scale compressed. Next, as shown in FIG. 11 A, embodiment 
5000 (TSM System 5400 of embodiment 5000 shown in FIG. 12) time-expands the 
first 5 time units of the work and creates a new data stream 3220 which has duration 10 
units and playback rate of 1/2. Note that only 5 time units of the beginning of the 
original audio or audio-visual work are time-scale expanded. As further shown in FIG. 
11 A, playback of either of these two data segments 3210 and 3220 may begin at the 
midpoint of the interval between re-broadcast start times (RBI/2), and, upon reaching 
the end of each Time-Scale Modified data stream, the client will be at the same 
segment location of the audio or audio-visual work being re-broadcast on an earlier or 
later offset data stream, respectively. In the preferred embodiment, segments (labeled 
3211, 3212) of the single time-scale compressed leader 3210 are used to merge with 
the previous offset stream from starting times between (n)RBI and (n)(RBI/2) as shown 
in FIG. 11 A. Similarly, segments (labeled 3221, 3222) of the single time-scale 
expanded leader 3220 are used to merge with the next offset stream from starting times 
between (n)(RBI/2) and RBI(n+l) as shown in FIG 1 1 A. 

[00087] As is readily apparent from FIG. 11 A, in accordance with the present 
invention, further, smaller time-scale compressed and time-scale expanded portions of 
the audio or audio-visual work are re-broadcast from times between the midpoint of the 
re-broadcast interval, and merge with the offset streams. In the preferred embodiment 
segments of the time-scale compressed leader and segments of the time-scale expanded 
leader are broadcast from starting times other than the offset stream start-times and 
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merge with the offset streams. The interval between re-broadcasts of the Time-Scale 
Modified Leaders is Leader Re-broadcast Interval (LRBI), which LRBI can be selected 
by embodiment 5000. hi accordance with a preferred embodiment of the present 
invention, as shown in FIG. 11 A, re-broadcasts of the time-scale compressed leader 
5 and time-scale expanded leader may be accomplished utilizing the technique of Time- 
Division Multiplexing for further efficiency. 

[00088] Advantageously, in accordance with this aspect of the present invention, 
there is no need to compute playback rates, and a media server needs only: (a) to select 
a Time-Scale Modified Leader to send to the client and (b) to manage a transition from 

10 the Time-Scale Modified Leader to a data stream being transmitted at normal speed 
when appropriate. Thus, in accordance with this aspect of the present invention, SRD 
4400 and VRB 4600 of embodiment 4000 are replaced with a Time-Scaled Leader 
Duration Determiner, Time-Scale Modification apparatus, and apparatus for streaming 
and multicasting the Time-Scale Modified Leaders. In accordance with the preferred 

15 embodiment of the present invention, the Time-Scale Modified Leaders are re- 
broadcast at regular intervals separated in time by an amount LRBI. 
[00089] Although the inventive technique has been described using two time- 
scale modified leaders and segments thereof, the invention is not thusly limited and 
embodiments using leaders with unique TSM rates (playback rates) are possible. In 

20 addition, multiple TSM leaders with different start times and different TSM rates may 
share a common intercept point in the work. 

[00090] FIG. 1 IB shows, in graphical form, encoding portions of a Time-Scale 

Modified audio or audio-visual work to form Time-Scale Modified Leaders. Playback 
time of segments (SegO, Segl, SegN) of an original audio or audio visual work are 

25 plotted along time axis 4037. These segments are encoded as data in accordance with 
methods that are well known to those of ordinary skill in the are, and the transmission 
times for work-encoded-data-blocks (DO, Dl, DN) which correspond to the 
segments (SegO, Segl, SegN) are shown along time axis 4039. Playback time of 
segments (SegO, Segl, SegN) for a time-scale compressed leader (compresses by a 

30 factor of 2) are shown along time- axis 4041. These segments are encoded as data, and 
the transmission times for work-encoded-data-blocks (DO, Dl, DN) which 
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correspond to the segments (SegO-Segl, Seg2-Seg3, ....) are shown along time axis 
4043. Lastly, playback time of segments (SegO, Segl, SegN) for a time-scale 
expanded leader (expanded by a factor of 2) are shown along time-axis 4045. These 
segments are encoded as data, and the transmission times for work-encoded-data- 
blocks (DO, Dl, DN) which correspond to the segments (SegO/2, Seg0/2, Segl/2, 
Segl/2, ....) are shown along time axis 4047. 

[00091] FIG. 12 shows a block diagram of embodiment 5000 of the present 
invention which: (a) transmits Time-Scale Modified Leaders; (b) joins re-broadcast 
offset streams of an audio or audio-visual work; and (c) transmits offset streams of an 
audio or audio- visual work. As shown in FIG. 12, Re-broadcast Interval Determiner 
(RJBID) 5700 receives, as input: (a) a code (for example, a number) which represents 
the duration of an audio or audio-visual work being re-broadcast and (b) a code (for 
example, a number) that represents the number of re-broadcast offset channels. RBID 
5700 produces, as output, data representing the duration of the re-broadcast interval 
("RBI"). The RBI is applied as input to Work Streamer 5200, Leader Re-broadcast 
Interval Determiner 5710, and Time-Scaled Leader Duration Determiner ("TSLDD") 
5600. In accordance with the present invention, RBID 5700 computes the RBI by 
applying one of a number of formulas. In the preferred embodiment, the following 
formula is used: 

RBI = Duration of audio or audio-visual work / No. of re-broadcast offset channels (7) 
[00092] Work Streamer 5200 and Multicaster 5300 are identical to Work 
Streamer 4200 and Multicaster 4300, respectively, which were described above with 
respect to embodiment 4000 shown in FIG. 10. 

[00093] As further shown in FIG. 12, TSLDD 5600 receives as input: (a) RBI 
(from RBID 5700); (b) a user defined parameter representing the time-scale 
compression (or speed-up) rate to use; and (c) a user defined parameter representing 
the time-scale expansion (or slow-down) rate to use. In response, TSLDD 5600 
produces, as output, the maximum time interval of the input audio or audio-visual 
work that will be required for creation of time-scale-compressed leaders and time-scale 
expanded leaders (applied as input to TSM System 5400). TSLDD 5600 computes the 
time interval of the original audio or audio-visual work that will be time-scale 
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compressed to form a time-scale compressed leader and the time interval of the 
original audio or audio-visual work that will be time-scale expanded to form a time- 
scale expanded leader by applying the following formulas: 
[00094] Speed-up: 

Tdo = (I72)(Speed/(Speed -1)) (8) 
[00095] where: Tdo = Time Interval of the original audio or audio-visual work 

to be time-scale compressed; Speed = speed-up factor (i.e. > 1) = 1 /time-compression 
factor; and I = Re-broadcast interval. 
[00096] Slow-down: 

Tdo = (I/2)(Speed/(l-Speed)) (9) 
[00097] where: Tdo = Time Interval of the original audio or audio-visual work 
to be time-scale expanded; Speed = slow-down factor (i.e. < 1) = 1 /time-expansion 
factor; and I = Re-broadcast interval. 

[00098] As one can readily appreciate from the above, for speed-up by a factor 

of 2 (i.e., time-compression by 1/2) and slow-down by a factor of 1/2 (i.e., time- 
expansion by 2), the time-compressed leader is obtained from a segment of the original 
audio or audio-visual work which starts at the beginning of the audio or audio-visual 
work and has a time interval equal to the re-broadcast interval. The time-expanded 
leader is obtained from a segment of the original audio or audio-visual work which 
starts at the beginning of the audio or audio-visual work and has a time interval equal 
to 1/2 the re-broadcast interval. This can also be understood as described above with 
respect to FIG. 11 A. 

[00099] As shown in FIG. 12, Time-Scale Modification System (TSMS) 5400 

receives as input: (a) data from Storage Device 5100 representing the original audio or 
audio-visual work; (b) the time interval of the original audio or audio-visual work 
required to generate time-scale compressed leaders (from TSLDD 5600); (c) the time 
interval of the original audio or audio-visual work required to generate a time-scale 
expanded leaders (from TSLDD 5600); (d) the speed factor for time-scale compression 
(from TSLDD 5600); and (e) the speed factor for time-scale expansion (from TSLDD 
5600). TSMS 5400 produces as output (a) a time-scale compressed leader; and (b) a 
time-scale expanded leader. TSMS 5400 time-scale compresses (at the specified speed 
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factor) the specified duration of the original work to produce the time-scale 
compressed leader. Similarly, TSMS 5400 time-scale expands (at the specified speed 
factor) the specified duration of the original work to produce the time-scale expanded 
leader. In a preferred embodiment of the present invention, the time- scale-compressed 
leader and the time-scale expanded leader are stored in Storage Device 5100. 
[000100] Storage Device 5100 is a storage device of a type which is well known 
to those of ordinary skill in the art. Storage Device 5100 stores a representation 
(preferably a digital representation or any representation that can be converted to a 
digital representation in accordance with methods which are well known to those of 
ordinary skill in the art) of an audio or audio-visual work or data of interest to a user 
(such as a stream of stock quotes, market data, advertisements, and so forth) and Time- 
Scale Modified Leaders. 

[000101] As shown in FIG. 12, Request Processor (RP) 5500 receives, as input, 
requests for data from clients connected via a network (for example, an Internet, WAN, 
LAN, or the like). In response, Request Processor 5500 produces, as output, 
information identifying the client and the appropriate re-broadcast control information 
such as, for example, "request for data that is identified by an appropriate data 
identifier," "disconnect," and other messages that are used to obtain data from 
embodiment 5000. It should be clear to those of ordinary skill in the art, that such 
information identifying the client and appropriate re-broadcast control information may 
be obtained: (a) by dialogs between the client and Request Processor 4500 in 
accordance any one of many methods that are well known to those of ordinary skill in 
the art including, without limitation, by use of forms that are contained on web pages 
that are transmitted to the client over a network in accordance any one of the many 
methods that are well known to those of ordinary skill in the art. Output from RP 5500 
is applied as input to Stream Assignment System (SAS) 5550. 

[000102] As shown in FIG. 12, SAS 5550 receives, as input: (a) information from 
RP 5500; (b) leader-offset- stream information (described in detail below) from TSCL 
Streamer 5210; (c) leader-offset-stream information (described in detail below) from 
TSEL Streamer 5220; and (d) information (described in detail below) from LRBID 
5710. SAS 5550 produces, as output, control information which is received by 
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Multicaster 5300, TSCL Multicaster 5310, and TSEL Multicaster 5320. In accordance 
with this embodiment of the present invention, SAS 5550 first determines a temporally 
closest leader-offset-stream by computing distances, forward and backward, in time 
from the arrival time of a client's request to view an audio or audio-visual work, to the 
5 previous Time-Scale Modified Leader-offset-stream start time and the next Time-Scale 
Modified Leader-offset-stream start time, and choosing the smaller of the two as the 
temporally closest leader-offset-stream. This can be performed by either of two 
methods: (a) by monitoring information output from TSCL Streamer 5210 and TSEL 
Streamer 5220 or (b) by computing the start times for the leaders from information 

10 provided by LRBDD 5710. In accordance with this embodiment of the present 
invention, SAS 5550 then produces, as output, information that directs either TSCL 
Multicaster 5310 or TSEL Multicaster 5320 to add the requesting client to the list of 
destinations for the appropriate Time-Scale Modified Leader offset stream segments 
being re-broadcast. SAS 5550 also sends information to Multicaster 5300 and either 

15 TSCL Multicaster 5310 or TSEL Multicaster 5320 which information conveys client 
identification and control information and the "intercept-time" for the corresponding 
Timq-Scale Modified Leader offset stream to an offset stream of the original audio or 
audio- visual work. In response to the intercept information, TSCL Multicaster 5310 
and TSEL Multicaster 5320 note the intercept time and schedule the deletion of the 

20 requesting client from the list of multicast recipients of that TSM Leader offset stream 
after the intercept time. Additionally, and in response to the intercept information, 
Multicaster 5300 notes the intercept time and schedules the addition of the requesting 
client to the list of multicast recipients of that offset stream after the intercept time. 
[000103] As shown in FIG. 12, Time-Scale Expanded Leader Streamer 5220 

25 receives, as input: (a) a code (for example, a number) which represents a desired 
Leader Re-broadcast Interval ("LRBI") from Leader Re-broadcast Interval Determiner 
5710; (b) a code (for example, a number) which represents a desired number of Time- 
Division Multiplexed (TDM) channels for the Time-Scale Expanded Leader from 
Leader Re-broadcast Interval Determiner 5710; (c) a code (for example, a number) 

30 which represents the duration of the leader being re-broadcast from Leader Re- 
broadcast Interval Determiner 5710 (alternatively, this information could have come 
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directly from TSLDD 5600); (d) the start times of the re-broadcast offset streams of the 
original audio or audio-visual work from Leader Re-broadcast Interval Determiner 
5710; and (e) data for the audio or audio-visual work from Storage Device 5100. 
Time-Scale Expanded Leader Streamer 5220 produces, as output: (a) the Time- 
Division Multiplexed composite signal of leader segments in a manner similar to that 
described above for Work Streamer 4200 (applied as input to TSEL Multicaster 5320) 
and (b) a stream of information giving the playback position and time-offset for each 
time-offset re-broadcast stream of the Time-Scale Expanded Leader (applied as input 
to SAS 5550). 

[000104] As shown in FIG. 12, Time-Scale Compressed Leader Streamer 5210 
receives, as input: (a) a code (for example, a number) which represents a desired 
Leader Re-broadcast Interval (LRBI) from Leader Re-broadcast Interval Determiner 
5710; (b) a code (for example, a number) which represents a desired number of Time- 
Division Multiplexed (TDM) channels for the Time-Scale Compressed Leader from 
Leader Re-broadcast Interval Determiner 5710; (c) a code (for example, a number) 
which represents the duration of the leader being re-broadcast from Leader Re- 
broadcast Interval Determiner 5710 (alternatively, this information could have come 
directly from TSLDD 5600); (d) the start times of the re-broadcast offset streams of the 
original audio or audio-visual work from Leader Re-broadcast Interval Determiner 
5710; and (e) data for the audio or audio-visual work from Storage Device 5100. 
Time-Scale Compressed Leader Streamer 5210 produces, as output: (a) the Time- 
Division Multiplexed composite signal of leader segments in a manner similar to that 
described above for Work Streamer 4200 (applied as input to TSCL Multicaster 5310) 
and (b) a stream of information giving the playback position and time-offset for each 
time-offset re-broadcast stream of the Time-Scale Compressed Leader (applied as input 
to SAS 5550). 

[000105] As shown in FIG. 12, Leader Re-broadcast Interval Determiner 
(LRBID) 5710 receives, as input: (a) a code (for example, a number) which represents 
the duration of the time-scale compressed and time-scale expanded leaders from 
TSLDD 5600; (b) a code (for example, a number) which represents the number of 
leader re-broadcast offset channels Re-broadcast Interval Determiner 5700; and (c) a 



-37- 

code (for example, a number) which represents the RBI from RBID 5700. LRBID 
5710 produces, as output, data which represents the duration of the Leader Re- 
broadcast Interval ("LRBI") (applied as input to TSCL Streamer 5210 and TSEL 
Streamer 5220). Leader Re-broadcast Interval Determiner 5710 computes the LRBI by 
5 applying the following formula: 

LRBI = RBI of audio or audio-visual work / No. of leader re-broadcast offset channels 
(10) 

[000106] Note that the greater the number of leader re-broadcast offset channels, 
the shorter the LRBI, and the less time a requesting client must wait to begin receiving 

10 media data from embodiment 5000. 

[000107] As shown in FIG. 12, TSCL Multicaster 5310 receives, as input: (a) a 
composite signal from TSCL Streamer 5210 and (b) client information (for example, 
control and destination) from SAS 5550. Likewise, as shown in FIG. 12, TSEL 
Multicaster 5320 receives, as input: (a) a composite signal from TSEL Streamer 5220 

15 and (b) client information (for example, control and destination) from SAS 5550. 
Then, in accordance with the present invention, TSCL Multicaster 5310 and TSEL 
Multicaster 5320 produce, as output, data (for example, message packets) directed 
toward particular clients for re-broadcast on a network such as the Internet, WAN, 
LAN, etc. In accordance with the present invention, TSCL Multicaster 5310 and TSEL 

20 Multicaster 5320 each manages a list of all clients that should receive data from 
particular portions of the TDM composite signal in accordance with any one of a 
number of methods which are well known to those of ordinary skill in the art. Then, 
whenever the particular portion of data in the TDM composite signal is received from 
TSCL Streamer 5210 and TSEL Streamer 5220, respectively, TSCL Multicaster 5310 

25 and TSEL Multicaster 5320, respectively, sends the particular portion of data to all 
clients (recipients) in the list who are to receive the particular portion of data (also 
known as multicasting). Many methods for broadcasting a portion of data from a data 
stream (for example, a Time-Division Multiplexed composite signal) to multiple 
recipients are well known to those of ordinary skill in the art. Control information 

30 from SAS 5550 is used in accordance with methods that are well known to those of 
ordinary skill in the art to modify the list of recipients, for example, to add a recipient, 
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or to remove a recipient from the list of destinations when the recipient no longer 
desires to receive data from the server. Lastly, whenever the data stream for a leader 
merges with a re-broadcast offset stream, the client is removed from the list of 
recipients. 

[000108] Embodiment 5000 has been described in detail above using separate 
components for ease of understanding the present invention, however, it should be 
clear to those of ordinary skill in the art that many of the components perform similar 
functions to their counterparts in embodiment 4000. Further, many of these 
corresponding components may be combined without loss of functionality. Still 
further, the intercept intervals in embodiment 5000 of the present invention follow a 
regular and periodic pattern which advantageously simplifies calculations of slope or 
playback rate and can be easily implemented using standard techniques such as lookup 
tables, count-down timers, and the like, all of which are well known to those of 
ordinary skill in the art. 

[000109] Conversely, components of embodiment 5000 may exist in separate 
locations connected to one another via a network or any other communication means 
(where the use of the term means is used in the broadest sense possible). 
[000110] Although FIG. 12 shows embodiment 5000 to be comprised of separate 
modules, in a preferred embodiment, the modules are embodied as software programs 
or modules which run on a general purpose computer such as, for example, a personal 
computer. It should be well known to one of ordinary skill in the art, in light of the 
detailed description above, how to implement these programs or modules in software. 
[000111] Those skilled in the art will recognize that the foregoing description has 
been presented for the sake of illustration and description only. As such, it is not 
intended to be exhaustive or to limit the invention to the precise form disclosed. 
[000112] Advantageously, embodiments of the present invention may be used to 
distribute Movies, Documentaries, or other audio and/or audio-visual works 
electronically, such as, for example, pay-per-view programs, video rental and the like. 
For example, although the distribution was described using networks, it should be 
understood that the term network is used in the broadest sense of the word and includes 
local distribution over cable, for example, a hotel distribution system, distribution over 
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satellite, distribution over the public airwaves where terminals are used in accordance 
with well known methods to authorize client access. It should also be clear that 
Request Processor 5550 can include functionality (in accordance with methods that are 
well known to those of ordinary skill in the art) to charge money for receiving works 
such as movies, sporting events and so forth. 

[000113] For example, those of ordinary skill in the art should readily understand 
that whenever the term "Internet" is used, the present invention also includes use with 
any non-deterministic delay network. As such, embodiments of the present invention 
include and relate to the world wide web, the Internet, intranets, local area networks 
("LANs"), wide area networks ("WANs"), combinations of these transmission media, 
equivalents of these transmission media, and so forth. 

[000114] In addition, it should be clear that embodiments of the present invention 
may be included as parts of search engines used to access streaming media such as, for 
example, audio or audio-visual works over the Internet. 

[000115] In further addition, it should be understood that although embodiments 
of the present invention were described where the audio or audio-visual works were 
applied as input to playback systems, the present invention is not limited to the use of a 
playback system. It is within the spirit of the present invention that embodiments of 
the present invention include embodiments where the playback system is replaced by a 
distribution system, which distribution system is any device that can receive digital 
audio or audio-visual works and re-distribute them to one or more other systems that 
replay or re-distribute audio or audio-visual works. In such embodiments, the playback 
system is replaced by any one of a number of distribution applications and systems 
which are well known to those of ordinary skill in the art that further distribute the 
audio or audio-visual work. It should be understood that the devices that ultimately 
receive the re-distributed data can be "dumb" devices that lack the ability to perform 
Time-Scale modification or "smart" devices that can perform Time-Scale 
Modification. 

[000116] Although the present invention has been described using Time-Scale 
Modified Leaders to catch-up or slow-down to a re-broadcast offset stream of a media 
work played at a normal playback rate (slope = 1), the present invention is not thusly 



-40- 

limited. For example, further embodiments of the present invention can be utilized to 
enable clients to merge with time-scale compressed versions of a work (slope > 1), 
time-scale expanded versions of a work (slope < 1), or to enable clients to migrate 
from a time-scale modified version of a work at one particular playback speed to a 
time-scale modified version of the same work with a different playback speed. In 
accordance with the present invention, this is accomplished by providing transitions 
from one data stream to another at specific intercept points, or by employing time-scale 
modified leaders to transition between versions of a work with different playback 
speeds. 

[000117] In accordance with the present invention, these embodiments are 
fabricated using embodiment 5000 described above, with a modification to SAS 5550. 
The modification enables SAS 5550 to compute the temporally closest stream with the 
requested time-scale (i.e., playback-rate) requested by the client at any point during 
playback. 

[000118] FIG. 13 shows a graph of location (offset from an origin) in normal and 
Time-Scale Modified versions of offset re-broadcasts of an audio or audio-visual work 
versus time on the horizontal axis. As shown in FIG. 13, nine (9) offset data streams 
are being re-broadcast. Three, (13010, 13110, 13210), correspond to re-broadcast of 
the work with normal playback rate (slope = 1.0); three, (13020, 13120, 13220), 
correspond to re-broadcast of the work with time-scale compression by a factor of 2 
(slope = 2.0); and three, (13005, 13105, 13205), correspond to re-broadcast of the work 
with time-scale expansion by a factor of 2 (slope = 1/2). As further shown in FIG. 13, 
at each re-broadcast interval, three (3) stream broadcasts are initiated (compressed, 
normal, expanded) and that these offset data streams intercept one another at regular 
intervals where the lines cross (lines intersect when the playback locations and 
playback times are equal). Clients wishing to view or listen to a work at a different 
rate can make a seamless transition to the new rate at points of intersection because the 
playback locations at the intersections are identical in each of the streams. As still 
further shown in FIG. 13, transition leader 13001 (normal playback rate, slope = 1) is 
shown to demonstrate a transition from playback stream 13005 (one-half normal 
playback rate, slope = 1/2) to playback stream 13020 (twice normal playback rate, 



-41- 

slope = 2.0). Additionally, a time-scale modified transition leader 13002 (twice normal 
playback rate, slope = 2.0) is used to transition from playback stream 13105 (one-half 
normal playback rate, slope = 1/2) to playback stream 13110 (normal playback rate, 
slope = 1) at a time before the normal intersection with playback stream 13220. 
5 Embodiment 5000 can be used to generate the re-broadcasts and the time-scale 
modified leaders to enable the client to traverse the re-broadcast matrix shown in FIG. 
13. In that case, client requests will be received by Request Processor 5500 and 
transmitted in the manner described above with respect to the description of 
embodiment 5000. 

10 [000119] For purposes of clarity and ease of understanding the present invention, 
the foregoing detailed description has used constant time-scale modification factors, 
but the present invention is not thusly limited and includes the use of varying time- 
scale modification factors using the same method and apparatus described above. 
Thus, Time-Scale Modification may be varied with time without loss of generality. In 

15 this case, the duration and slope of such a continually varying Time-Scale Modified 
signal can be computed using any one of a number of formulae, including the 
following formula: 

Duration = previous duration + (time-interval * tsm factor) (11) 
[000120] Although for clarity and ease of understanding the previous inventions 
20 have been described broadcasting a single work, it should be clear that embodiments of 
the present invention are not thusly limited and the inventive technique can be applied 
to multiple works existing in a single embodiment. 

[000121] Another aspect of the present invention relates to the use of extra 
information broadcast in conjunction with an audio or audio-visual work from a server 

25 to restrict, or direct, playback rates at the client. Embodiments of this aspect of the 
present invention can be used in a variety of ways. For example, a public service 
announcement regarding emergency information, safety information, and the like may 
be missed if the client is listening at a very fast rate (learning impaired and hearing 
impaired individuals may wish to have important public service or emergency 

30 broadcasts played at playback rates below the normal playback rate to aid in 
comprehension). Since these messages may be of vital importance, a need exists to 
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restrict the playback rate for the client in a client-server system that supports Time- 
Scale Modification, or to notify the client of the importance of these messages. 
[000122] In accordance with this aspect of the present invention, information 
used to restrict, or direct, playback rates at the client may be "in-band" (i.e., occurring 
5 within the signal being transmitted, for example, as a specific frequency tone or data 
code within a header of a data packet) or "out-of-band" (i.e., occurring within a data 
packet associated with, but not comprising, media data). 

[000123] FIG. 14 shows a block diagram of embodiment 21000 of the present 
invention which transmits information relating to the content and/or appropriate 

10 playback speed of media data to clients receiving the media data. 

[000124] As shown in FIG. 14, Streaming Data Source 2 1 1 00 provides, as output: 
(a) data representing an audio or audio-visual work through Network 21200 (Network 
21200 is a network in the broadest sense described above), there are many methods 
which are well known to those of ordinary skill in the art for fabricating Streaming 

15 Data Source 21100, and (b) a signal to TSM Control Source 21150 indicating the 
transmission of the data and client identifier information for clients who will have 
playback rates restricted. Note that the components of embodiment 21000 may exist in 
separate locations connected to one another via a network or any other communication 
means (where the use of the term means is used in the broadest sense possible). 

20 [000125] As further shown in FIG. 14, in response to the signal, TSM Control 
Source 21 150 transmits playback code data, for example, a code, on Network 21200 to 
all identified clients that will have their playback rates directed for data received from 
Streaming Data Source 21 100. 

[000126] Although FIG. 14 shows the transmission of the media data and the 
25 playback code data being transmitted over the same network, the present invention is 
not thusly limited. In fact, the present invention includes embodiments where the 
media data and playback codes are transmitted over different communications paths. 
Further, the transmission of the playback code need not be coordinated with the 
transmission of the media data as described above. The playback code data can be 
30 transmitted prior to the transmission of the media data and include information used to 
enable its coordination with the media data. For example, the playback code data may 
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include times (and time intervals) in the media work that are targeted for use in 
restricting playback rates. 

[000127] As still further shown in FIG. 14, Capture Buffer 21400 receives the 
following, as input: (a) media data input from Network 21200; (b) requests for 
5 information about the amount of data stored therein from a Capture Buffer Monitor 
(not shown, this is optional); and (c) requests for media stream data from TSM System 
21800. In response, Capture Buffer 21400 produces the following, as output: (a) a 
stream of data representing portions of an audio or audio-visual work (applied as input 
to TSM System 21800); (b) a stream of location information used to identify the 

10 position in the stream of data (applied as input to TSM System 21800); and (c) an 
indication of the amount of data stored therein (applied as input to the optional Capture 
Buffer Monitor). It should be well known to those of ordinary skill in the art that 
Capture Buffer 21400 may include a digital storage device. There are many methods 
well known to those of ordinary skill in the art for utilizing digital storage devices such 

15 as, for example, a "hard disk drive," to store and retrieve general purpose data and there 
exist many commercially available apparatus which are well known to those of 
ordinary skill in the art for use as a digital storage device such as, for example, a CD- 
ROM, a digital tape, a magnetic disk, and so forth. TSM Control Decoder 21450 
receives, as input, playback code data from Network 21200 (as discussed above, it can 

20 receive this data from a different communication channel as well). TSM Control 
Decoder 21450 produces, as output, playback code data and applies it, as input, to 
TSM Rate Determiner 21700. 

[000128] As yet still further shown in FIG. 14, TSM Rate Determiner 21700 
receives the following, as input: (a) a signal (from TSM Control Decoder 21450) that 

25 represents a playback rate; (b) client generated playback rate requests (received by a 
client interaction interface apparatus in accordance with any one of many methods 
which are well known to those of ordinary skill in the art (not shown for ease of 
understanding the present invention)); (c) a parameter designated Interval_Size; and (d) 
a parameter designated SpeedChangeResolution. In response, TSM Rate Determiner 

30 21700 produces, as output a rate signal representing a TSM rate, or playback rate, 
which rate signal is applied as input to TSM System 21800. TSM Rate Determiner 
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21700 generally passes client generated rate requests to its output (without 
modification), thereby enabling the client to control the playback rate. Whenever TSM 
Rate Determiner 21700 receives information from TSM Control Decoder 21450, TSM 
Rate Determiner 21700 processes that information to determine an appropriate 
playback rate to output. For example, TSM Rate Determiner 21700 may override the 
client requested rate request if the information from TSM Control Decoder 21450 
specifies that the media work being received is of critical importance to public safety. 
Additionally, TSM Rate Determiner 21700 may process the information from TSM 
Control Decoder 21450 according to a rule-set or other algorithm specified by the 
client. For example, the rule-set or algorithm can fast-forward through all commercial 
content in the media work being received, slow down for specific types of content, 
using techniques described in U.S. Patent Application entitled "Method and Apparatus 
for Generation of Listener Interest Filtered works, which patent application has the 
same inventor as the present application and is incorporated by reference herein. 
[000129] In a preferred embodiment of the present invention, TSM Rate 
Determiner 21700 uses a parameter Interval_Size to segment the input digital data 
stream in Capture Buffer 21400 and to determine a single TSM rate for each segment 
of the input digital stream. Note, the length of each segment is given by the value of 
the Interval Size parameter. Further, TSM Rate Determiner 21700 uses a parameter 
Speed_Change_Resolution to determine appropriate TSM rates to pass to TSM System 
21800. A desired TSM rate is converted to one of the quantized levels in a manner 
which is well known to those of ordinary skill in the art. This means that the TSM 
rate, or playback rate, can change only if the desired TSM rate changes by an amount 
that exceeds the difference between quantized levels, i.e., Speed_Change_Resolution. 
As a practical matter then, parameter Speed_Change_Resolution filters small changes 
in TSM rate, or playback rate. The parameters Interval_Size and 
Speed_Change_Resolution can be set as predetermined parameters for embodiment 
21000 in accordance with methods which are well known to those of ordinary skill in 
the art or they can be entered and/or varied by receiving user input through a user 
interface in accordance with methods which are well known to those of ordinary skill 



-45- 

in the art. However, the manner in which these parameters are set and/or varied are not 
shown for ease of understanding the present invention. 

[000130] Embodiments of TSM System 21800 and Playback System 21500 have 
been described in detail above. 
5 [000131] Although FIG. 14 shows embodiment 21000 to be comprised of 
separate modules, in a preferred embodiment, the modules are embodied as software 
programs or modules which run on a general purpose computer such as, for example, a 
personal computer. It should be well known to one of ordinary skill in the art, in light 
of the detailed description above, how to implement these programs or modules in 
10 software. 

[000132] Conversely, components of embodiment 21000 may exist in separate 
locations connected to one another via a network or any other communication means 
(where the use of the term means is used in the broadest sense possible). 
[000133] Advantageously, in accordance with the present invention, the above- 

15 described information used to control the playback speed during particular portions of 
the media work, for example, commercials. Thus, a client may be prevented from fast 
forwarding through the commercials. However, in an alternative embodiment, clients 
may pay for the ability to automatically fast-forward through all commercial 
advertisements in an audio or audio-visual work being received, hi such an alternative 

20 embodiment, a user interface module (fabricated in accordance with any one of the 
many methods which are well known to those ordinary skill in the art) receives client 
requests and charge information to effectuate the functionality. 

[000134] FIG. 15 shows a block diagram of embodiment 22000 of the present 
invention in which information relating to the playback speed and/or content of the 

25 media data being broadcast to clients is embedded in the media work. As shown in 
FIG. 15, Media Work Source 22100 provides, as output, data representing an audio or 
audio- visual work to Compositer 22160 transmits playback code data, for example, a 
code, to Compositer 22160. Compositer 22160 outputs a data signal which is encoded 
by Encoder 22170 using any number of methods well known to those of ordinary skill 

30 in the art and transmitted to Network 22200 (Network 22200 is a network in the 
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broadest sense described above) to clients, for example, who have requested the audio 
or audio visual work. 

[000135] As further shown in FIG. 15, data from Network 22200 is applied as 
input to Decoder 22310 which decodes the encoded data signal using any number of 
methods well known to those of ordinary skill in the art. The decoded data is applied 
as input to Separator 22320 which separates the playback code data from the media 
data and applies the playback code data as input to TSM Control Decoder 22450 and 
the media data as input to TSM System 22800. TSM Control Decoder 22450 decodes 
the playback code information and produces as output a TSM Rate which is applied as 
input to TSM Rate Determiner 22700. hi accordance with the present invention, TSM 
Rate Determiner 22700, TSM System 22800 and Playback System 22500 are the same 
as the corresponding components described above with respect to embodiment 21000. 
In accordance with this embodiment of the present invention, TSM Rate Determiner 
22700 uses the TSM Rate from TSM Control Decoder to set the playback rate and 
override client input. 

[000136] Although FIG. 15 shows embodiment 22000 to be comprised of 
separate modules, in a preferred embodiment, the modules are embodied as software 
programs or modules which run on a general purpose computer such as, for example, a 
personal computer. It should be well known to one of ordinary skill in the art, in light 
of the detailed description above, how to implement these programs or modules in 
software. 

[000137] Conversely, components of embodiment 22000 may exist in separate 
locations connected to one another via a network or any other communication means 
(where the use of the term means is used in the broadest sense possible). 
[000138] Although the detailed description used the terms playback rate and TSM 
rate, and the terms playback and playback apparatus, these terms should be understood 
to include any type of presentation rate (i.e., a rate of presentation of information) and 
any type of presentation apparatus. As such, these terms are to be understood as being 
used in the broadest sense. In addition, although the detailed description used the 
terms media, media work, media data, media broadcast, audio or audio-visual work, 



-47- 

and information, these terms should be understood to refer to any type of information 
or data. As such, these terms are to be understood as being used in the broadest sense. 



